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Large sample asymptotics for the 
two-parameter Poisson— Dirichlet process 

Lancelot F. James* ^ 

Hong Kong University of Science and Technology 

Abstract: This paper explores large sample properties of the two-parameter 
{a, 9) Poisson-Dirichlct Process in two contexts. In a Bayesian context of es- 
timating an unknown probability measure, viewing this process as a natural 
extension of the Dirichlet process, we explore the consistency and weak conver- 
gence of the the two-parameter Poisson-Dirichlet posterior process. We also 
establish the weak convergence of properly centered two-parameter Poisson- 
Dirichlet processes for large 9 + net. This latter result complements large 9 
results for the Dirichlet process and Poisson-Dirichlet sequences, and com- 
plements a recent result on large deviation principles for the two-parameter 
Poisson-Dirichlet process. A crucial component of our results is the use of 
distributional identities that may be useful in other contexts. 
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1. Introduction 

In this work, for < a < 1 and 9 > —a, we are interested in the two-parameter 
class of random probability measures that are formed by 

oo k — 1 

(1-1) Po.A-)^J2^kl[{i-v,)6zA-) 

k=i j=i 

where the Vk are independent beta(l — a, 6+ka) random variables and, independent 
of these, the {Zk) are an i.i.d. sequence with values in some Polish space / with com- 
mon (non-atomic) distribution H. That is to say Pa.e is a random probability mea- 
sure taking values in Vi, where Vj is the set of all probability measures on /. We will 
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simply say that a random probability measure P is a two-parameter (a, 0) Poisson- 
Dirichlet process, having law say Wa.e on Vi, suppressing dependence on H, if P can 

be represented as in (1.1). That is P(-) = Pa,e{-)- For shorthand we write P - liafi. 
We denote the expectation operator corresponding to the law liafi as Ea^e, which 
is such that Eafi[P{-)] = H{-). We note that the Vk are obtained by size-biasing 
a ranked sequence of probabilities known as the two-parameter Poisson-Dirichlet 
sequence. It follows that by permutation invariance Pafi may also be represented 
in terms of this sequence. Many properties of the two-parameter Poisson-Dirichlet 
sequence, as it primarily relates to Bessel and Brownian phenomena, were discussed 
in [3G]. This sequence has gained in importance as it is seen to arise in a number 
of different areas including, for instance, Bayesian statistics, population genetics 
and random fragmentation and coalescent theory connected to physics. See [34] for 
some updated references and [19] for some connections to Dirichlet means. 

When (2 = then P is a Dirichlet process in the sense of Ferguson [9] . Pitman [35] 
showed that within a Bayesian context, these random probability measures can be 
seen as natural and quite tractable extensions of the Dirichlet process. In particular 
Pitman [35] showed that if random variables Xi, . . . , X„ given P are i.i.d. P and P 
has prior distribution Hafi^ then the posterior distribution of P|Xi, . . . , X„, denoted 
as n[^"g, corresponds to the law of the random probability measure, 

P!^ei') = Rn(p)Pa,e+n{p)ai-) + (1 - ^ri(p))Ai(-) 

where 

n(p) 

D„(-)-E^'^^.(-)- 

(Ai, . . . , A„(p)) is a Dirichlet(ei — a, ... , en{p) ~~ ct) random vector. All the random 
variables appearing on the right hand side are conditionally independent given the 
data. P„(p) is a heta{6 + n{p)a, n— n(p)a) random variable and {Yi, . . . , ^^(p)} de- 
notes the 1 < n(p) < n unique values of {Xi, . . . , Xn-} Furthermore, Pa,e+7i{p)a ^ 
^aM+n{p)a- Sj IS the number of Xi equivalent to Yj for j — 1, . . . , 7i(p)., When 
a = one obtains the posterior distribution derived in Ferguson [ >]. The notation 
Cj and n{p) are taken from Lo [29], as discussed in Ishwaran and James [i / ]. Fur- 
thermore, a generalization of the Blackwell-MacQueen [4] prediction rule is given 

by 



Note also that 
We also write 



p(x„+ie • lXi,...,x„) = ii;[p^5(.)]. 



^(p) 



n — n[p)a 

which importantly reduces to the empirical distribution when a = 0, or when 
n(p) = n. 

Let Pq°° denote a product measure on /°° making Xi for i = 1, . . . , oo indepen- 
dent with common (true) distribution Pp. In this paper, extending the known case 
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of the Dirichlet process, we show that as n ^ oo the posterior distribution Il)^g 

(n) 

behaves as foUows. When Pq is discrete then 11^ g converges weakly to a point mass 

at Pq a.s. P^. When Pg is continuous, that is, non-atomic, 11^""^ converges weakly 
to a point mass at the mixture aH + (1 — a)Po. Thus when Pq is discrete/atomic 
the posterior distribution is consistent. However, when Pq is non-atomic the poste- 
rior distribution is inconsistent, unless either a = which corresponds to the case 
of the Dirichlet process, or more implausibly one chooses _ff = Pq. In addition to 
this result we establish a functional central limit theorem, for the case where Pq 
is non-atomic, by showing that the process P^"j centered at its expectation (1.2), 
indexed over classes of functions, converges weakly to a Gaussian process. This is in 
line with nonparametric Bernstein- Von Mises results of for instance, [31], [30], [28], 
[5] and [22]. Additionally, we note that the weak convergence of the two-parameter 
Poisson-Dirichlet process may be of interest in other fields. In particular, in order 
for us to discuss the posterior weak convergence, when Pq is non-atomic, we will 
need to address the weak convergence of the centered process 

as n oo, which poses additional challenges. Note this process does not depend on 
the data, except through the sample size n. Furthermore, the study of Va.e+na is 
more in line with the literature on the behavior of Dirichlet processes and Poisson- 
Dirichlet sequences when 6 oo. See, for instance, [21], [7] and [32]. Additionally, 
our work is complementary to a recent result of [8] on large deviation principles for 
the two-parameter Poisson-Dirichlet process. 

Returning to the consistency result, in terms of estimating the true Pq in a non- 
parametric statistics setting, our result shows that unlike the case of the Dirichlet 
process, it is perhaps unwise to use Ila,e as a prior. However we should point out 
that Ishwaran and James [17, 18], owing to the attractive results in [35], suggested 
that one could use Ila,e in a mixture modeling setting analogous to the case of the 
Dirichlet process in Lo [29]. In this more formidable setting one can deduce strong 
consistency of the posterior density, induced by the priors Ha.e on the mixing dis- 
tribution, by using the results of Lijoi, Priinster and Walker [2")]. In fact, in a work 
subsequent to ours, this was recently shown by Jang, Lee and Lee [20]. We also note 
that those authors also obtain our consistency result as a special case. For some 
more results on the modern treatment of Bayesian consistency in nonparametric 
settings one may note, for instance, the works of Ghosal, Ghosh and Ramamoorthi 
[11], Barron, Schervish and Wasserman [2] and Ghosal, Ghosh and van der Vaart 
[12]; and the book of Ghosh and Ramamoorthi [13]. 



2. Consistency 

This section describes the consistency behavior of the posterior distribution in the 
case where the true distribution Pq is either continuous or discrete. First we note 
the following fact; 

Lemma 2.1. Let f and g denote measurable functions on I, then for < a < 1 
and 9 > —a, 

EaAP{f)P{9)] = ^-^H{g)H{f) + ]-^H{fg). 
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Proof. The proof proceeds by using disintegrations. First the joint distribution of 
{Xl, P) can be written as, 

P{dxi)Il^,e{dP) = IicAdP\^i)H{dxi) 

where na,e{dP\xi) Tl^^\{dP). Then, 



P{dx2)Iio..e{dP\xi) = Tl'^^\{dP)E[P{dx2)\Xi = x^ 



(2') 

where W^Q{dP) ~ Tlo,.e{dP\xi,X2) and 



E[P{dx2)\Xi = Xl] = tt^H{dx-2) + \-^5.Mx2). 



Now this gives 

Ec,AP{dxi)P{dx2)] = H{dxi) 



ct , s 1 — a „ , 

— H{dx2) + j^S,,,id^2) 



which by writing P[g)P{f) ~ Jj Jj g{xi)f{x2)P{dxi)P{dx2) completes the result. 

□ 

We proceed as in Diaconis and Freedman[10] by showing that the posterior dis- 
tribution concentrates around the prediction rule. First using Diaconis and Freed- 
nian ([10], p. 1117), we define a suitable class of semi-norms such that convergence 
under such norms implies weak convergence. Let A ~ Ui^i be a partition of /. 
Then define the semi-norm between probability measures 



(2.1) 



P-Q\a^ 



^[p(A,)-g(A,)]' 



for a suitable generating sequence of partitions A where, naturally for any Q, 
Si^iQ(Ai) = 1- Now similar to ([10], Equation 14) for the Dirichlet process, 
we will show that the posterior distribution concentrates around the prediction 
rule (1.2). 

In order to do this one only needs to evaluate the posterior expectation, express- 
ible as, 



(2.2) 



E 



P' 



where E[P^g] equates with the prediction rule probability given in (1.2). We obtain, 
Lemma 2.2. 



E 



Proof. First using basic ideas we expand, for each set A. 

(p("J(AO-i?[Pi"i(A,)])'. 

Furthermore, 



|2 



< 



n + 1 



(A,)I)„(A.) + (l-^n(p)r^^(A.) 
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Now from Lemma 2.1 



Additionally, 



2 1 ^ (9 + n{p)a)ie + n{p)a + 1) 



pr.-, p (n-n(p)a)(n-7i(p)a + 1) 



and 



It follows that 



{e + n){e + n + i) 

E[AiA 



(e-, - a){ei - a) 
(ri — n(p)Q!)(n — n(p)Q; + 1) 



pr/p(")/ . _ (g + n(p)a)(g + 7i(p)Q + Q) . ^ 



{e + n){e + n + l) 
{n~n{p)af 

{e + n){e + n + i) 



Now 



,(")^4.M^' - (g + n(p)a 



,2 



(0 + n) 



+ ?i) 
(n-7i(p)a)'^ 
[0 + n) 

Taking differences and using the fact that J^t^i -^n{^i)H{Ai) < 1 and similar 
arguments completes the result. □ 

(n) 

Proposition 2.1. If Pq is continuous then the posterior distribution 11^ g converges 
weakly to point mass at the distribution 

aH{-) + {l^a)Po{-) a.e. . 

Hence the posterior is consistent only if either P is a Dirichlet process or H = Pq. 

Proof. From Lemma 2.2 it follows that the posterior distribution must concentrate 
around the prediction rule. Now, under Pq (assuming a continuous Pq) the predic- 
tion rule becomes, 

(2.3) P(X„+i e . . . . , X„) = '-^H{.) + ± 6^^ (.). 

It is clear that under Pq, that P{Xn+i G ■ \Xi, . . . , X„) converges uniformly over 
appropriate Glivenko-Cantelli classes to 

aH{-) + il-a)Po{-) 

for almost all sample sequences. One gets this by simple algebra and classical results 
about empirical processes (the second term in (2.3)) appropriately modified. □ 
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The previous results says that the posterior distribution is inconsistent for aU 
non-atomic Pq unless a = or one has chosen H ^ Pq. The behavior in the case 
where Pq admits ties is quite different and is summarized in the next result 

Proposition 2.2. Suppose that Pq is a discrete law such that n{p)/n then the 

(n) 

posterior distribution 11^ g converges weakly to point mass at Pq, a.e. P^, for all 
< a < 1 and 6 > —a. 

Proof. Since, n{Tp)/n — ^ 0, it follows that 

(2.4) P(x„,, e . |x„ . . . , X.) . ^^^(0 + E (•), 

converges uniformly to Pq for almost all sample sequences Xi, X2, . . . ,. This is true 
since, 

j=i 1=1 j=i 

where the second term on the right converges to zero. □ 



2.1. Some more limits 

Note that one obtains some information on the limit behavior of the random prob- 
ability measure Pa,0+n(p)a which has law T^a,e+n{p)a- This type of result is more in 
line with large 6 type asymptotics. In this case large 9 is replaced by large n{p)a. 

Proposition 2.3. Suppose that Pq is such that n{p) — > 00, then the two-parameter 
Poisson-Dirichlet law ^a,0+n{p)a converges weakly to point mass at H. 

Proof. The proof proceeds by again utilizing the semi-norm in (2.1). We have that 
E'a,B+n(p)a[P] = H. Furthermore, from Lemma 2.1. the variance of P{A) under 
^a,e+n(p)a IS, for cach A, equal to, 

-H{A){1~H{A)). 



6 + 7j{p)a + 1 
Hence if P has law T^a.e+n{p)ai then, 

n i2 1 1 — ck 

i?a,«+„(p)a [\P-H\a\ < e + „(p)«+l' 

completing the result. □ 



3. Bernstein von Mises and functional central limit theorems 

In this section we address the more formidable problem of establishing functional 
central limit theorems for centered versions of the two-parameter process. We will 
restrict ourselves to the case where Pq is continuous which presents some difficul- 
ties. In particular, in that setting, we are interested in the asymptotic behavior in 
distribution, as n — ^ 00, of the posterior process 
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conditional on the sequence Xi, X2, . . ., and the asymptotic behavior of the process, 

uniform over classes of functions J-. For clarity, wc first mention some elements 
of the (modern) framework of weak convergence of stochastic processes on general 
function indexed Banach spaces. There is a rich literature on this subject, here we 
use as references [39], [.38], Ch. 10, and [14]. Let denote a collection of measurable 
functions / : / ^ 3? and let 1°°{!F) denote the set of all uniformly bounded real 
functions on T . Now for a random probability measure (5„, and the probability 
measure defined as its expectation E[Qn] Q, wc consider the maps from —f ^ 
given by the linear functional 

f ^Qnif)= Jj{x)Qnidx) 

and 

f ^ Q{f) = Jj{x)Q{dx). 

— \/n{Qn—Q){-) denotes its centered process and let Gg denote a Gaussian 
process with zero mean and covariances 

(3.1) E[GQ{f)GQ{g)] = Qifg) - Q{f)Q{g). 

A Gaussian process with covariance (3.1) is said to be a Q-Brownian bridge. We will 
assume that !F possesses enough measurability for randomization and write, as in 
([37], p. 2056), T G M{Q). The notation L2{Q) represents the equivalence class of 
square integrable functions. Furthermore, a function F{x) such that |/(a;)| < F{x) 
for all X and / € is said to be an envelope. 

We are interested in the cases where the sequence of processes {G'„(/) : / £ 
G (T) converges in distribution to a Gaussian process Gq uniformly over !F. 
In that case we write 

G„ -Gq in i^iT). 

Furthermore, because we are interested in the convergence of posterior distribu- 
tions, if Gn depends on data Xi, X2 . . ., we will need the more delicate notion of 
conditional weak convergence of G„(-) for almost all sample sequences Xi,X2 . . . , 
and we write 

Gn Gq in e°°{T) a.s. . 

More formally, one may say that the processes converge in the sense of a bounded 
Lipschitz metric outer almost surely (see [14] and [39]). 

The rich theory of weak convergence of empirical processes addresses the cases 
where Qn = Pn = Y^^=i (•) is the empirical measure, and Q = Ep^ [P„] = Pq 
is the true underlying distribution of the data. Hence one has, 

(3.2) ^{Pn - Po) ^ Gpo in 

for many classes of T. The classes are said to be Po-Donsker. The classical case 
of the empirical distribution function, Fn{t) = '^"=1 1{Xi<t)/Ti arises by setting 
ft{x) = l(-cx3<x<t) for t ranging over 3?; see ([3n], Example 19.6). 

The most notable results for convergence conditionally on the data, center around 
the bootstrap and its exchangcably weighted extensions where 

00 

Qn{-) := Pw{-) 
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for [Wi) an exchangeable sequence of positive weights summing to 1. In particular, 
we will use the following result of ([HT], Theorem 2.1), 



provided that (3.2) holds, Pii[F'^) < oo, and the weights satisfy certain conditions 
as given in [37]. The constant c is determined by the weights. The result generalizes 
the result of [IC] for Efron's bootstrap empirical measure. In the case of Efron's 
bootstrap and the Bayesian bootstrap c = 1. For results on the real line see [3], [27] 
and [33]. 

As mentioned at the beginning of this section, we will consider the behavior of 

(n) 

the process g in the case where Pq is continuous. We shall see that we can handle 

part of the weak convergence of u'^g by utilizing results in [37]. This is very much 
in the spirit of [27], [26] and [5]. However we will also need to deal with the behavior 
of the process h'a,ff+na- This process is considerably more challenging to handle as 
it is not obviously related to an empirical-type measure. However, in Section 4, we 
will exploit an important distributional identity that allows us to treat Va.e+na, as 
a measurelike sum of i.i.d. processes in the sense of [1], [39], Section 2.11.1.1, and 
[41]. Throughout we will assume that T S M{H) and that there exist an envelope 
F{x) satisfying H{F^) < oo. 

We shall also have need of the (unconditional) multiplier central limit theorem 
on Banach spaces for i.i.d. random variables in the Lorcntz 7^2,1 space, which is 
found in [24]. More details may be found in [39] (see also [15] and [14]). A random 
variable f (see [39], Section 2.9) is said to be in ^2,1 if 



Finiteness of ||C||2,i requires slightly more than a second moment but is implied by 
a 2 + e absolute moment. In our case, the L2,i condition is easily satisfied as the 
variables we shall encounter are gamma random variables having all moments. 

3.1. Continuous case 

(n) 

We now address the case of weak convergence of g when Pq is continuous. Here we 
will need to consider the process i'a,d+na- Because, we will not be working strictly 
with empirical-type processes, but actually measure-like processes, we will restrict 
ourselves to the quite rich class of J- that constitute a Vapnik-Chcrvoncnkis graph 
class (VCGC) (see for instance [41], p. 239). This avoids the need to otherwise state 
uniform entropy-type conditions. We now present the result below for i'^"]. We will 
present a partial proof of this result and then address the behavior of i'a,e+na, in 
the next section. 

Theorem 3.1. Let T be a VCGC subclass of L2{Po) and L2{H) with envelope F 
such that Pq{F'^) < 00 and H{F^) < 00. For < a < 1, and 6 > -a, let ^'^"e(•) 

denote the posterior, n|^"g, process centered at its mean and scaled by ^Jn. Then 
when Pq is continuous, conditional on the sequence Xi, X2, ■ ■ ■ , 



(3.3) 



V^iPw - Pn) cGp„ in £°°{T),a.s. 




V 



Vl~aGp„ + y/a{l - a)<GH + V^N{Po - H) in £°°{T) 



a.s. 



Where G p„ and <Gh are independent Gaussian processes, independent of N which 
is a standard Normal random variable. 
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Proof. The process v^^g is equivalent to 



+ na 



■n 



e + n 



■na • 



Now since Pq is continuous it follows that Fn equates with the empirical measure 
P„(-) = Sr=i = where Pw has weights 



where Gi-a^i are i.i.d. gamma(l — a, 1) random variables. Furthermore, i?„ is 
beta(0 + na,n{l — a)) and hence converges in probability to a as 7i — > oo. So 
the first term is asymptotically equivalent to the process 

VTi{l - a){Pw - Pn) 

and it follows that the result of ([-Si], Theorem 2.1) applies. Hence the process, 
without the (1 — a) term, satisfies (3.3) with c = 1/^/1 — a. It is easy to see that 
Rn centered by its mean and scaled by y/n converges to a Normal distribution, 
hence the second term converges in distribution to y/aN{H — Pq). Finally, the limit 
of Va.e+na will bc vcrificd in the next section. □ 

4. Asymptotic behavior of a Poisson— Dirichlet (a, 6 + ncx.) process 

In this last section we establish the weak convergence of the process Va^e+na- Since 
Pa,e+na is closcly associated with various properties of Brownian and Bessel pro- 
cesses we expect that this result will be of interest in those settings. The establish- 
ment of this result requires a few non-trivial maneuvers as Pa.e+na does not appear 
to have any similarities to an empirical process. We first establish an important 
distributional identity 

Proposition 4.1. Let Pa.e+na denote a random probability measure with law 
^a,e+na, then 



where Pa,a Oure i.i.d. 11^ q, random probability measures independent of Gg, {Ga^i), 
where Ga,i are i.i.d. gamma{a, 1) random variables, Gg is gamma{9, 1), independent 

of iGa,i) and Gg+na = Gg + J2i=l Ga.i- 

Proof. It is enough to establish this result for Pafi+naig), for an arbitrary bounded 
measurable function g. The result follows by noting that for any 9 > 0, 



which is equivalent to the Cauchy-Stieltjes transform of order 9 of Pa,g{g), whose 
form was obtained by [40]. It follows that, 




7ia 
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which is the same as 



-AG„,,pW^(g)i 



i=l 

Thus we can conclude that 

n 

1=1 

Now, using the fact that Gg+na = Gg + Ga,i is gamnia(0 + na, 1), it follows 

using the calculus of beta and gamma random variables that 



Gg-\-na-Pa,g-]-na 

(•) Gg+r, 



„ - U.I/ \ / 1 / J ^ 



where on the right hand side, Gg+„a is independent of the term in brackets. Now we 
use the fact that gamma random variables are simplifiable to conclude the result. 
See Chaumont and Yor ([G], Sections 1.12 and 1.13) for details on simplifiable 
random variables. □ 

Proposition 4.1 now allows us to write 



l'a,g+na — V n- 



Additionally one has for ^^ > 0, the covariance formula 

cov{^{P^S - ^{P.fi - H){g)) ^ ^-^[H{fg) H{f)H{g)], 

which follows from Lemma 2.1. Using these points we obtain the next result. 

Theorem 4.1. Let T he a VCGC subclass of L2{H) with envelope F such that 
H{F^) < oo. Let, for < a < 1 and 6 > —a, Pa,g+na denote the random 
probability measure with Poisson-Dirichlet law Ila^g^na, and define Va.g+na{') '■= 
V^iPa.g+na ~ H)^-)- Then as oo, 

Va,e+na ^ ^ r-^ '&H in (T) . 

Proof. The key to this result is of course Proposition 4.1, which allows us to express 
i^a,g+na, m tcrms of i.i.d. processes Pa}a having mean H and variance for each A, 
as 

\^HiA)[l-HiA)]. 
1 + a 

In particular, it follows that the asymptotic distributional behavior of t'a.e+na is 
equivalent to that of 

n „ 

V^^Y^^±^ip(%-H)i■). 

1=1 ^j=l "-^"'^ 

Appealing, again, to the law of large numbers we may instead use the process 



1 ^G^(pW _^)(.) 

y/n ^ a 
^ i—i 
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which decomposes into the sum of two asymptotically independent pieces, 
^ n „ 1 " 

^ %—\ ^ z— 1 

The second term is a mcasure-likc process in the sense of [ ] and [39], Section 
2.11.1.1. However since we have chosen the class to be VCGC, convergence of this 
process follows by using arguments similar to that in [41], Section 7.8. The first term 
then converges by the multiplier CLT in Banach spaces ([24] or [2.3], Proposition 
10.4). □ 

I^.l. Dirichlet process asymptotics for — > oo 

The next result describes weak convergence of a centered Dirichlet process as ^ 
oo. 

Theorem 4.2. Let T be a VCGC subclass of L2{H) with envelope F such that 
H{F^) < oo. Let, for 6 > 0, Po.e denote a Dirichlet Process with law llo,0, having 
mean H, and define Tg{-) :— ^/6(Pq — H)(-). Assume without loss of generality 
that 9 = riK, for k a fixed positive number. Then as n —f oo, 

Te Gh ini°°{T), 

where Gh is a H-Brownian bridge. Furthermore the limit does not depend on k. 
Proof. By an argument similar to Proposition 4.1, one can write 

n 
i=l 

Hence rg is asymptotically equivalent to 

^ i — l ^ ^ ^ i—1 

The result then follows analogous to Theorem 4.1. □ 
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